An Algorithm to Assess Guideline Concordance of Antibiotic Choice in Community-Acquired Pneumonia.

BACKGROUND AND OBJECTIVE: This study aimed to develop and evaluate an algorithm to reduce the chart review burden of improvement efforts by automatically labeling antibiotic selection as either guideline-concordant or -discordant based on electronic health record data for patients with community-acquired pneumonia (CAP). METHODS: We developed a 3-part algorithm using structured and unstructured data to assess adherence to an institutional CAP clinical practice guideline. The algorithm was applied to retrospective data for patients seen with CAP from 2017 to 2019 at a tertiary children's hospital. Performance metrics included positive predictive value (precision), sensitivity (recall), and F1 score (harmonized mean), with macro-weighted averages. Two physician reviewers independently assigned "actual" labels based on manual chart review. RESULTS: Of 1345 patients with CAP, 893 were included in the training cohort and 452 in the validation cohort. Overall, the model correctly labeled 435 of 452 (96%) patients. Of the 286 patients who met guideline inclusion criteria, 193 (68%) were labeled as having received guideline-concordant antibiotics, 48 (17%) were labeled as likely in a scenario in which deviation from the clinical practice guideline was appropriate, and 45 (16%) were given the final label of "possibly discordant, needs review." The sensitivity was 0.96, the positive predictive value was 0.97, and the F1 was 0.96. CONCLUSIONS: An automated algorithm that uses structured and unstructured electronic health record data can accurately assess the guideline concordance of antibiotic selection for CAP. This tool has the potential to improve the efficiency of improvement efforts by reducing the manual chart review needed for quality measurement.

Digital Antimicrobial Stewardship Decision Support to Improve Antimicrobial Management.

OBJECTIVES: We sought to create a digital application to support clinicians in empiric and pathogen-directed antibiotic ordering based on local susceptibility patterns and evidence-based treatment durations, thereby promoting antimicrobial stewardship. METHODS: We formed a multidisciplinary team that met bimonthly from 2017 to 2018 to design and construct a web-based antimicrobial stewardship platform called Antibiogram + . We used an iterative and agile technical development process with frequent feedback from clinicians. RESULTS: Antibiogram+ is an online tool, accessible via the electronic health record and hospital intranet, which offers institutional antibiotic susceptibilities for major pathogens, recommendations for empiric antibiotic selection and treatment durations for common pediatric conditions, antimicrobial dosing and monitoring guidance, and links to other internal clinical decision support resources. The tool was accessed 11,823 times with 492 average monthly views during the first 2 years after release. Compared with use of a preexisting print antibiogram and dosing card, pediatric residents more frequently reported "often" being sure of antibiotic dosing with Antibiogram+ (58 vs. 15%, p < 0.01). Respondents also reported improved confidence in choice of antibiotic, but this finding did not reach statistical significance (55 vs. 35%, p = 0.26). CONCLUSION: We report the successful development of a digital antimicrobial stewardship platform with consistent rates of access during the first 2 years following release and improved provider comfort with antibiotic management.

Design, Usability, and Acceptability of a Needs-Based, Automated Dashboard to Provide Individualized Patient-Care Data to Pediatric Residents.

BACKGROUND AND OBJECTIVES: Pediatric residency programs are required by the Accreditation Council for Graduate Medical Education to provide residents with patient-care and quality metrics to facilitate self-identification of knowledge gaps to prioritize improvement efforts. Trainees are interested in receiving this data, but this is a largely unmet need. Our objectives were to (1) design and implement an automated dashboard providing individualized data to residents, and (2) examine the usability and acceptability of the dashboard among pediatric residents. METHODS: We developed a dashboard containing individualized patient-care data for pediatric residents with emphasis on needs identified by residents and residency leadership. To build the dashboard, we created a connection from a clinical data warehouse to data visualization software. We allocated patients to residents based on note authorship and created individualized reports with masked identities that preserved anonymity. After development, we conducted usability and acceptability testing with 11 resident users utilizing a mixed-methods approach. We conducted interviews and anonymous surveys which evaluated technical features of the application, ease of use, as well as users' attitudes toward using the dashboard. Categories and subcategories from usability interviews were identified using a content analysis approach. RESULTS: Our dashboard provides individualized metrics including diagnosis exposure counts, procedure counts, efficiency metrics, and quality metrics. In content analysis of the usability testing interviews, the most frequently mentioned use of the dashboard was to aid a resident's self-directed learning. Residents had few concerns about the dashboard overall. Surveyed residents found the dashboard easy to use and expressed intention to use the dashboard in the future. CONCLUSION: Automated dashboards may be a solution to the current challenge of providing trainees with individualized patient-care data. Our usability testing revealed that residents found our dashboard to be useful and that they intended to use this tool to facilitate development of self-directed learning plans.